The Generating Function of Ternary Trees and Continued Fractions

count ternary trees and the number of certain plane partitions and alternating sign matrices. Tamm evaluated these determinants by showing that the generating function for these entries has a continued fraction that is a special case of Gauss’s continued fraction for a quotient of hypergeometric series. We give a systematic application of the continued fraction method to a number of similar Hankel determinants. We also describe a simple method for transforming determinants using the generating function for their entries. In this way we transform Somos’s Hankel determinants to known determinants, and we obtain, up to a power of 3, a Hankel determinant for the number of alternating sign matrices. We obtain a combinatorial proof, in terms of nonintersecting paths, of determinant identities involving the number of ternary trees and more general determinant identities involving the number of r-ary trees.